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A  characteristic  of  cultural  features  that  aDOear  in 
high-resolution  aerial  photographs  is  the  laroe  number  of  straight 
edges.  Usually,  these  edges  occur  in  pairs,  as  in  the  sides  of 
roads  ana  of  buildings*  As  a  o re l i m i na r y  step  towards  identifying 
such  features,  the  straight  edges  can  ce  extracted  (Nevatia  ana 
Sacu,  *973;  Peleg,  1 0  7  9  3  «  and  clustered  into  antiparallel  oairs 
li.e.  pairs  of  facing  edges  that  are  parallel  but  have  opposite 
directions).  The  clustering  process  is  the  subject  of  this 
report.  In  general,  clustering  must  take  into  account  information 
from  the  picture  in  the  regions  around  the  edges.  For  example,  a 
road  usually  has  a  uniform  gray  level  (Ouam,  157S,  Tavakoli,  in 
progress)  and  thus  it  is  reasonable  to  expect  the  facing  sides  of 
an  antiparallel  oair  of  edges  to  have  a  similar  gray  level.  The 
process  to  be  described  here,  however,  deals  only  with  a  line 
drawing  extracted  from  an  image,  and  does  not  have  information  of 
this  sort  available  to  it.  It  has  nevertheless  proved  itself  a 
rocust  and  reasonably  successful  techniaue,  despite  its  relative 
simplicity. 

The  process  accepts  as  input  ordered  oairs  of  endooints 
defining  lines  and  their  directions.  It  attempts  to  find  pairs  of 
lines  that  are  more  or  less  antiparallel  and  that  obey  relations 
concerning  overlap  and  distance  between  the  lines.  The  process 
considers  all  lines  that  have  similar  orientations,  and  uses  an 
iterative  procedure  to  find  good  pairs.  whereas  previous  «ork 
(Nevatia  and  Sabu,  1573,  9rooks,  1979)  has  restricted  the  choice 
of  pairs  to  lines  that  are  closest  neighbors,  the  current 
algorithm  makes  a  more  atooal  analysis. 

The  basic  procedure  is  as  follows.  Fach  line  is  compared 
with  all  other  lines  that  are  approximately  antiparallel  to  it. 


i.e.  those  that  lie  within  a  l eng t h-depencent  threshold.  The 


inversely  proportional  to  the  length  of  the  line*  so  that  short 
lines,  whose  directions  are  less  certain,  are  allowed  a  qreater 
discrepancy  than  longer  lines,  the  figure  of  merit  for  a  given 
pairing  of  lines  is  based  on  the  amount  of  overlap  between  the 
lines,  i.e«  the  proportion  of  common  length  of  their  projections, 
and  the  oistance  between  the  lines.  Several  ways  of  compining 
these  measures  were  evaluated,  and  the  results  inaicated  that  the 
exact  combination  was  not  critical  to  the  procedure. 

Having  given  initial  scores  to  the  pairs  of  lines,  the  neat, 
iterative,  step  involves  refining  the  scores  by  interactions  among 
the  tines  that  compete  for  the  same  antioarallel  partner,  Each 
line  has  a  linn  to  every  possible  antiparallel  pairing  line.  Each 
link  has  a  link  strength,  and  the  link  strengths  are  normalised. 
Should  a  line  have  no  links  to  other  lines  or  should  there  be  no 
links  to  it,  the  line  is  dropped  from  consideration.  1*  t«o 
antiparallel  lines  A  and  8  are  a  mutually  best  pair,  i.e.,  if  a's 
link  strength  to  9  is  stronger  than  its  link  strength  to  any  other 
line,  and  B's  link  to  A  is  also  its  stronoest  link,  and  there  are 
no  lines  between  A  and  9,  then  A  ana  B  are  considered  to  be 
linked  and  can  be  dropped  from  further  cons iae r a t i on »  Note  that 
when  this  happens,  any  other  lines  that  were  linked  to  A  or  to  9 
must  hove  these  links  deleted.  Because  of  the  deletions  a 
renormalization  is  reguired,  and  this  adjusts  the  link  strengths 
with  respect  to  alternative  anti-parallel  pairs  for  lines.  The 
process  is  iterated  until  no  more  pairs  can  be  formed.  figure  la 
shows  a  picture  of  part  of  an  airfield.  Figure  1b  shows  the  set 
of  directed  edges  derived  from  the  picture,  and  Figure  tc  shows 
the  results  of  applying  the  process  to  the  lines  in  Figure  la. 

The  rest  of  this  paper  describes  the  method  in  detail, 
presents  examples  of  its  application  to  sets  of  lines  extracted 
from  aerial  photographs,  and  discusses  some  of  the  shortcomings 
ano  advantages  of  the  approach. 
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1 )  Read  in  t  ►  e  data  f  v,  r  each  tine  in  t^e  for’'  ( x  1  ,  r  1  ,  x  Z  ,  f  Z ) 
-here  (x  .  ,  Y 1 >  are  the  co-oroinates  of  tne  first  endooint  and  ( x  Z  , 
t  2 )  arc  the  co-ordinates  of  the  second  enccoint  of  the  tint. 

Z  5  Initialise  tr.e  tir»«  between  each  Qualified  Lai  r  o'  lines. 
( S «.  e  ulowi) 


Z)  \cf »at i.e  each  lint's  set  of  links  ano  eliminate  any  line  «ith 
no  l  i  r.x  s  . 


<• )  Rfcceat  ste«s  5-7  until  no  lines  remain: 

5)  for  every  link*  say  fro#  tine  A  to  line  ~  ,  and  every  line  C 

t"at  foils  bet.een  A  and  2  with  its  greatest  link  strength  to  line 

~t  deciement  me  link  trow  A  to  a  oy  D's  link  strength  to  C  if  D 
falls  outside  A  anc  6 . 

;)  eliminate  lines  mat  are  mutually  best  pairs  and  that  have  no 
other  lines  Between  the#,  Sreak  the  linns  from  or  to  these  lines. 

7)  R e-no rta l  i < e  each  line's  set  of  link  strengths  jna  eliminate 

any  line  with  no  links  to  it  or  from  it. 


The  algorithm  starts  fcx  reading  in  the  co-ordinates  of  the 
enecoints  of  each  line.  The  slope  of  the  line  is  defined  by  the 
orjer  in  which  the  endpoints  appear.  The  length  of  the  line  is 
also  Calculated  at  this  ooint. 

For  each  line,  all  other  lines  th«,t  are  ant  i-par  a  l  l  e  l  it 
j no  fall  within  a  computed  angle  tolerance  are  examined  to  se*  if 
they  are  candidates  f or  pairing  witn  the  line.  The  angle 
tolerance  is  computed  ,$  the  sum  of  two  factors  Cl  and  C ’ «  wnere 
C  *  is  a  constant  (currently  25  degrees)  jnd  C  2  is  inversely 
proportional  to  the  longer  cf  the  two  lines  oeing  con$id«reo, 
"‘■is  prwvides  greater  laxity  in  interpreting  short  lines  wnos“ 


Ji  recti  on s  are  less  certain  than  lonqer  lines. 

,-tt  tines  ahich  .ass  the  angle  test  must  in  addition  satisfy 
the  further  criterion  that  their  overlap  ce  nonzero.  The  ovrtac 
is  computed  as  the  intersection  ot  the  objections  o<  the  lines 
onto  6  line  th„t  runs  oetween  them  and  has  an  angle  that  is  the 
mec,n  c.f  their  angles  modulo  ISO  degrees.  (Figure  Z  •  ) 


All  lines  passing  the  angle  and  ovarian  tests  are  linked,  .ith 
J  strength  commuted  as  3  function  of  their  cistance  from  each 
other  and  thtir  overlap.  The  distance  oefeen  lines  «  and  a  is 
comouted  as  the  peroenoicular  distance  petween  lines  that  pass 
through  the  midpoints  of  A  and  9,  anu  .hose  direction  is  the 
average  of  these  of  A  ana  6.  (Figure  2.)  The  dotted  lines  in  tr.e 
figure  have  slope  half. ay  between  that  of  line  a  and  line  c. 


FIGURE  3 


The  current  function  oeing  used  for  tne  link  strength  from  line 
to  line  o  is 
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5?ver»l  variants  of  these  factors  were  useo.  Come  of  the  variants 
that  -ere  trieu  are 

1.  strenoth  =  ( 2*over lac/ (lenqtht line  A3*lenoth(line  6))/distance 

2.  strength  =  ever  lac/ lengthf line  A)  «1/oistance 

3.  strength  =  2  »o  ve  r  l  a  c  /  (  l  e  ng  t  h  ( l  i  ne  A)  Menothlline  £>)  ♦ 

’/distance 

Uote  that  variants  1  and  3  are  symmetric,  -hile  variant  2, 
and  toe  variant  described  above  are  asymmetric.  Asymmetric 
functions  were  preferred  over  symmetric  functions  because  it  seems 
reasonable  that  a  snort  line  should  be  mere  strongly  attracted  to 
3  long  line  than  the  long  line  is  to  the  short  line.  The 
aljOrithm  proved  remarkably  insensitive  to  the  initial  link 
strength  calculation;  aifferent  schemes  usually  led  to  very 

simitar  results. 

At  this  point,  any  lines  without  links  are  deleted.  Next  the 
iterative  part  of  the  algorithm  is  entered.  Its  goal  is  to  find 

octimal  pairings  of  line  segments  cases  on  th»  initial  estimates 

ana  satisfying  the  constraints  that  each  line  belong  to  at  most 

one  pair  and  that  no  two  pairs  overlap.  (See  Figure  A  nher*  »B 
ano  CD  are  examples  of  mutual  pairings  -hich  are  ruled  out  by  the 
overtac  r equ i r ament ) . 


FI3U9E  4 


Each  lin*,  for  instance  from  line  A  to  line  £.  is  updated  in 
o  a  r  a  1  l  e  l  according  to  the  following  s  e  n  e  m  e  .  It  t  ►*  e  r  e  is  a  line, 
say  line  C«  Detween  A  a«j  £  ,  which  is  .ithin  the  angle  tolfenc* 
C  *  mentioned  above,  tnen  C's  areatest  lin*  strength  is  f  ouna ,  say 
to  line  a.  if  line  0  is  also  Between  a  ana  £,  nothing  is  do ne  . 
Itr.  e  r  .  i  s  e  *  A's  link  strength  to  9  is  decremented  by  C's  link 
strength  to  C.  After  all  the  links  have  teen  updated  in  this 
manner,  the  link  strengths  are  re-normu l i 2td»  for  all  lines  1  ana 
‘j,  if  -'s  greatest  lin*  strength  is  to  9  ana  9  '  s  qreat»st  link 
strength  is  to  A  anj  there  are  no  lines  between  «  and  ° ,  then  » 
and  c  ore  contidereo  to  be  paired  and  are  dropped  from  further 
c  ons  i  ue  r<,t  ion  .  At  tnis  point,  any  line  with  no  links  to  it  or 
from  it  is  deleted,  ana  a  new  iteration  is  started. 

The  fcet-een.  relation  is  calculated  as  follows  (Ficur*  To 
discover  whether  or  net  line  C  is  Detween  lines  A  and  9,  a  l  i  n  <» 
-hose  slope  is  the  average  of  the  Slopes  of  lines  A,  9,  and  C  is 
cassea  through  the  center  joints  of  each  of  these  lines.  (Eee  the 
dotted  lines  in  Figure  5.)  Line  C  is  between  lines  A  and  5  i 4  the 
intercept  on  some  (arbitrary)  a  *  i  s  of  the  dotted  line  through  C's 
center  is  between  the  intercepts  of  the  lines  through  A  ana  ?,  and 
i«  line  C  overlaps  both  A  and  3. 
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cX»!rPLES  A\’C  OlsCuSSICN 


The  algoritnm  presented  here  performs  competently  on  a 
variety  of  input  data.  This  is  especially  significant  when  it  is 
realised  that  no  use  is  made  of  "holistic"  patterns  that  miqht  Ce 
apparent  to  nun. a n s  -ho  examine  the  data.  Euch  rattems  a  r  i  s  <* 
oecausc  of  i n t e r p r e t a f i on s  imposed  cn  the  scene  that  rely  on 
assumptions  external  to  the  scene,  and  thus  not  availacte  to  t  rs  e 
program.  These  assumptions  vary  from  person  to  person,  and 
different  people  often  prefer  cifferent  pairings  in  the  same 
scene.  As  a  result,  it  is  not  possible  in  oeneral  to  specify  non 
»  ®  l  l  the  system  performs  unless  there  is  an  associated  context  for 
the  interpretation. 


- . 1  £  x  amp l e  s 

In  oroer  to  alleviate  this  problem,  a  series  of  digitized 
imoces  -as  used  as  a  source  of  lines  for  the  program.  The 
strai.,nt  ecges  in  the  images  were  extracted  using  an  iterative 
enhancement  technioue  (Peleg,  197S),  ano  the  resulting  lines  -ere 
used  as  input  to  the  pairing  program.  Figure  6  shows  one  such 
image  ano  the  set  of  lines  extracted  from  it.  The  only  contextual 
information  retained  from  the  images  is  the  direction  of  each 
line.  This  direction  is  such  that  the  image  is  lighter  to  the 
right  of  the  line  than  to  the  left.  It  -ill  fce  seen  that  even 
this  small  amount  of  knowledge  can  be  valuable  in  d i s amb i gua t i nq 
line  .airings. 

The  use  of  images  of  real  scenes  enables  an  accurate 
evaluation  of  the  pairings.  It  should,  however,  fce  clear  that 
humans  use  much  more  information  in  forming  pairs  than  is 
available  to  tne  progr-m. 

In  the  images  that  were  used,  good  pairs  ccrresoona  to  the 
ed9es  of  roaas  and  run. ays  and  to  the  sloes  of  buildings.  iany 
cultural  features  are  c ha r ac t e r i ze d  oy  antiparallel  pairs  of 


strai3nt  ecaes,  ana  it  is  a  use^l  step  in  interpreting!  t  he  inage 
to  be  able  to  discover  these  pairs.  because  the  edge  extraction 
or o cess  is  likely  to  treak  the  lines  into  segments,  ano  because  of 
t^e  existence  of  cross-roads,  a  program  that  attempts  to  linn  ^.c 
colli  near  lines  is  usej  to  extend  and  connect  segments  that  lie  on 
a  Straight  line.  This  program  will  oe  the  subject  of  another 
report.  The  following  simple  scheme  was  u  s  <*  a  to  link  cell  inear 
segments.  A  more  ambitious  scheme  is  described  by  Frcoks  ( T  ^  7 ;  )  . 

The  algorithm  starts  by  picking  a  line  to  .ork  on.  It  then 
finds  all  otner  lines  .hose  angle  is  s u f f i c i e n t l y  ' c l o s e  to  that  of 
the  chosen  l i^c.  (Currently,  within  25  aejrees).  These  lines  are 
the  initial  candidates  for  collinear  linking.  vost  of  the 
canoicates  are  weecec  out  immediately  py  a  constraint  reouirina 
centers  of  lines  to  lie  within  a  narrow  ribbon  centered  on  the 
chosen  line  (usually  3  to  5  pixels  wide.)  (figure  7). 


It  remains  to  link  up  lines  that  are  not  too  far  3cart.  This  is 
essentially  a  parallel  process.  If  tne  oac  between  tne  end  ooint 
of  one  line  ana  the  start  of  the  next  is  small  enough  (less  than 
1.5  times  the  maximum  jf  the  lines'  lengths)  the  lines  are  merged. 
If  not,  the  enapoint  becomes  the  end  of  the  linked  stement,  and 
the  start  point  tecomes  the  start  of  a  new  segment.  Because  the 
length  that  is  used  is  not  that  of  the  whole  new  line,  hut  only 
that  of  its  final  segment,  the  results  do  not  depend  on  which  iin» 
is  chosen  first.  The  ^airing  program  receives  as  input  t*e  result 
of  applying  tht  collinearity  orogram  to  the  enhanced  data. 


figure  ca  sho-s  a  Dicture  of  a  part  of  Lorton  reformatory  in 
the  Fort  jclwoir  region  of  Virginia  for  which  Figure  ft  is  tne 
corres^ondin,  edge  image.  Figure  ca  sho.s  the  result  of  applyino 
the  Cullinear  algorithm,  while  Figure  8b  shows  tne  result  of 
a op lying  the  program  described  in  Section  7.  Notice  especially 
ro.  the  lines  comprising  the  cross-shared  hui l d i no  in  the  tower 
left  auaorant  of  the  picture  have  been  paired.  The  program  has 
erroneously  'ciund  the  shadow  borders,  insteao  of  those  o'  the 
ouiloir.o.  F  ic^re  cc  sho.s  ho.  a  very  small  amount  of  contextual 
information  c.n  te  use  f  u l  in  correcting  this  error.  The  extra 
information  is  that  builaings  usually  appear  brighter  than  tne  i  r 
surroundings.  !t  is  manifested  as  a  further  constraint  nlac»a  on 
the  pairing  process.  This  constraint  is  tnat  a  line  may  only  pair 
with  another,  an t i pa r a l l e l ,  line  if  the  intervening  region  is 
lighter  than  the  area  on  the  other  side  of  the  line.  Edges  are 
extracted  . itn  a  airection  corresponding  to  tne  convention  that 
the  lighter  side  of  the  edge  be  to  the  right.  Thus,  this 
constraint  me«,ns  that  3  line  is  only  allowed  to  pair  with  otner 
lines  that  are  to  its  riqht.  Using  this  constraint,  the  algorithm 
correctly  discovers  the  corders  of  the  building.  Fiqure  9  shows 
another  example  of  apolying  the  program  to  data  obtained  from  a 
suburban  scene  In  both  cases,  almost  all  pairs  are  correct,  and 
most  of  the  noise  lines  are  not  paired.  The  resultant  description 
is  much  more  useful  for  further  processing  than  the  orioinal  edge 
i mage  • 

I  •  Z  extensions 

The  pairing  algorithm  in  this  paper  can  be  criticized  for 
toning  too  local  a  view  in  determining  tne  best  matches.  It  night 
happen  that  a  pair  of  lines  that  are  mutually  best  partners  should 
not,  intact,  be  linkeu.  This  can  happen  if  the  result  o*  linkino 
them  is  to  cause  other  lines  with  low  pairing  strengths  to  be 
linked.  This  would  give  a  low  global  Pairinq  confidence,  whereas 
forming  alternative  pairings  might  lead  to  a  higher  global 


Anti-parallel  linking 
brightness  constraint 


Naive  anti-parallel  linking 
applied  to  (b) • 


Figure  9a.  A  suburban 
scene. 


b.  Lines  extracted  from  the  picture 
in  (a)  . 


Collinear  linking  of  lines 


d.  Anti-parallel  linking. 


confidence. 

4  way  of  alleviating  this  problem  is  to  cn.rae  the  pairinc 
process  from  a  discrete  process  to  a  f.n>  cne  and  to  use  a 
relaxation  algorithm  to  find  the  cest  pairs.  Instead  of  simply 
pairin',  lines  that  are  Mutually  best  links,  ano  celetinq  all  other 
links,  it  »Owld  be  oreferatte  to  increase  or  decrease  the 
confidence  in  each  link  according  to  the  mutual  compatibilities  of 
the  links  with  one  another.  3y  iterating  this  procedure,  a  more 
.lot ally  consistent  result  should  emerge. 

♦ 

Z  . o  alternatives 

Several  alternatives  to  the  pairing  procedure  were  considered, 
rut  were  discarded  in  favor  of  the  method  a?scr iceo  above.  Two  of 
the  alternative  methods  are  briefly  described  celow. 

The  first  alternative  has  t*»e  advantage  of  starting  from  a 
gradient  or  edge  image,  or  even  from  a  raw  ima^e,  without  havioa 
to  co  an,  preprocess inM.  It  finus  candidate  antiparallel  pairs  a* 
ed3es,  but  does  not  decide  which  are  the  best  wairinqs. 

The  idea  is  to  search  out  in  opposite  directions  from  each 
point  in  the  image  array,  looking  for  eage  responses.  By 
requiring  the  responses  to  be  at  the  same  distance  on  either  side 
of  me  coint  and  to  have  opposite  gradients,  a  measure  o*  how 
close  the  point  is  to  the  axis  of  the  antiparallel  lines  can  be 
obtained.  This  measure  should  deoend  on  the  (distance  that  has  to 
c?  searched  to  fine  a  pair  of  responses. 

The  result  of  applying  this  process  at  each  point  in  the 
image,  in  parallel,  is  a  set  of  responses  that  take  the  for-  o* 
linear  ridges  centered  between  pairs  of  straight  lines.  ’’he  se 
riuges  encoue  the  distance  and  overlap  information  for  the 
surrounding  lines,  on  a  po int -by-po int  basis.  They  can  be  used  as 
input  to  a  system  similar  to  the  current  one. 

The  advantages  of  this  method  are  that  it  extracts  skeletons 
cf  those  edges  that  are  oart  of  «r.t  ipar  at  le  l  Pairs,  and  that  it 
calculates  properties  of  the  pairs  that  are  useful  in  (later) 


*  r  oc  e  s  s  i  no .  it  suffers  from  flaws,  ho  wever,  which  nak*  it 
JOubt  tul  «liet  n?r  the  method  is  as  useful  as  the  edge  extraction 
ana  enhancement  process  that  was  actually  used. 

The  main  flaw  is  the  inaoility  of  the  method  to  find  all 
possible  pairs  without  further  processing.  Consider  the  situation 
in  Figure  1"  where  the  solio  lines  denote  actual  rones  and  the 


dotted  lines  denote 

axes  of  pairs. 
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The  axes  correspond 

to  the  pairs  AB, 

?c , 

and 

CD 

How eve r , 

t  he 

pair  AC  is 

also 

valid, 

but  is  not 

represented 

because  it  s 

axis 

falls  c e  t  ween 

lines 

fi  ana 

C.  D y  examining 

all 

axes  that  have 

the 

same  slope,  it  is  possible  to  discover  the  pairs  that  -ere 
initialy  missfw,  but  this  could  ce  a  costly  process.  delates  work 
on  shape  skeletons  (w«nq,  Uu  ana  Rosenfelo,  illustrates  the 
difficulties  of  applying  this  kind  of  local  processing. 

The  second  alternative  involves  associating  confidences  with 
pairs  Cased  on  similarities  of  features,  and  not  just  on  relative 
si/e  and  position.  Instead  of  calculating  a  merit  function  for 
each  pair  based  on  these  features,  a  set  of  nough-like  transforms 
can  Ce  applied  by  projecting  onto  various  subspaces  of  the  feature 
space.  This  allows  clusters  of  an t i pa r a l l e I s  to  be  detected, 
instead  of  having  to  seal  with  the  lines  cairwise. 

Unfortunately,  experience  has  shown  that  cluster  detection  in 
nough  spaces  can  be  very  difficult  (Rosenfeld,  13?C).  !«  the 
linear  feature  domain,  work  ty  irooer  et  al.  (in  progress)  seems 
to  indicate  a  lack  o*  significant  clusters  in  Hough  space. 


The 


Hcuqh-line  metnoc  thus  seems 
a-vocotea  in  this  (.acer* 


4.  CONCLUSIONS 


This  payer  has  presented  an  atoorithm  for  cair  ino 
jnc  i  p  a  r  a  l  lei  straight  lines.  The  pairing  .as  based  on  the  a'oant 
of  ovtriap  between  t*e  lines,  the  distance  tetween  then,  ano  on 
whether  or  not  other  lines  were  interposed. 

The  algorithm  wjs  evaluated  using  lines  entracter!  fro*  real 
images  using  an  eoge  detection  ana  enhancement  process.  Tne 
results  were  seen  to  be  generally  reasonable  and  the  result  of 
aoplying  the  process  was  a  new  image  representation  that  was  mere 
useful  for  further  processing  than  the  initial  eoge  image. 
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